1. Field of the Invention
The present invention relates to the field of consumer electronic devices. More specifically, the present invention relates to methods and systems for providing user interfaces for networked electronic devices including remote devices.
2. Related Art
The typical home entertainment system today consists of a variety of different consumer electronic devices which present and record audio/visual media in different ways. In the field of media devices, there is a spectrum of features for products of a given class (VCRs, video camera, etc.). Most of the features are represented by physical controls or elements on a control panel on the device which can be manipulated by a human user.
For instance, typical home audio-visual (AV) equipment includes a number of components such as a radio receiver or xe2x80x9ctuner,xe2x80x9d a compact disk (CD) player and/or a digital video disc player (DVD), a number of speakers, a television, a video cassette recorder (VCR), a tape deck, and the like. Each of these components is connected to the others via a set of wires. One component is usually the central component of the home AV system; for example, the tuner. The tuner has a number of specific inputs for coupling the other components. Tuner-based control is often limited to connecting/disconnecting another component""s AV input or output. The tuner has a corresponding number of control buttons or control switches which provide a limited degree of controllability and interoperability for the components. However, only rarely does a tuner control the component""s operation (e.g., rewind, play, etc.). The control buttons and control switches are usually located on the front of the tuner providing the user with a so-called hard front panel interface. In many cases, some, or all, of these buttons and switches are duplicated on a hand-held remote control unit, another type of hard front panel interface. A user controls the home AV system by manipulating the buttons and switches on the front of the tuner or, alternatively, by manipulating buttons on the hand-held remote control unit. This conventional home AV system paradigm has become quite popular.
As consumer electronic devices become more capable and more complex, the demand for the latest and most capable devices has increased. As new devices emerge and become popular, the devices are purchased by consumers and xe2x80x9cpluggedxe2x80x9d into their home AV systems. As a consumer purchases new devices (e.g., digital audio tape recorders, digital video disk players, digital camcorders, and the like), most often the new device is simply plugged into the system alongside the pre-existing, older devices (e.g., cassette tape deck, CD player, and the like). The new device is plugged into an open input on the back of the tuner or into some other device coupled to the tuner. The consumer (e.g., the user) controls the new device via the control buttons on the tuner, via the control buttons and control switches on the front of the new device itself, or via an entirely new and separate remote control unit for the new device.
As the number of new consumer electronic devices for the home AV system have grown and as the sophistication and capabilities of these devices have increased, a number of problems with the conventional paradigm have emerged. One such problem is incompatibility between devices in the home AV system. Consumer electronic devices from one manufacturer often couple to an AV system in a different manner than similar devices from another manufacturer. For example, a tuner made by one manufacturer may not properly couple with a television made by another manufacturer.
In addition, where one device is much newer than another device, additional incompatibilities may exist. For example, a new device might a incorporate hardware (e.g., specific inputs and outputs) which enables more sophisticated remote control functions. This hardware may be unusable with older devices within the AV system. Also, for example, older tuners may lack suitable inputs for some newer devices (e.g., mini-disc players, VCRs, etc.), or may lack enough inputs for all devices of the system. Another problem is the lack of functional support for differing devices within an AV system. For example even though a television may support advanced sound formats (e.g., surround sound, stereo, etc.), if an older and less capable tuner does not support such functionality, the benefits of the advanced sound formats can be lost.
Another problem is the proliferation of controls for the new and differing devices within the home AV system. For example, similar devices from different manufacturers can each have different control buttons and control switch formats for accomplishing similar tasks (e.g., setting the clock on a VCR, programming a VCR record a later program, and the like). In addition, each new device coupled to the AV system often leads to another dedicated remote control unit for the user to keep track of and learn to operate.
To address the problems above, a home AV network has been proposed for consumer electronic devices, including computing devices (e.g., personal computers and peripheral devices such as printers). The home AV network would provide a set of services which facilitate device interoperability and allow the devices to cooperate to perform application tasks. By connecting consumer electronic devices in a home AV network, it is also possible to share processing and storage resources. In addition, a home AV network allows consumer electronic devices to be coordinated and simultaneously controlled by a user, thereby simplifying that aspect of operation from the user""s perspective. The underlying structure of a home AV network consists of a set of interconnected clusters of devices. Typically, there may be more than one cluster in a home, perhaps one per floor or one per room. Each cluster functions as a set of interconnected devices to provide services to a single user or to multiple simultaneous users.
Recently, a class of consumer electronic media devices has been proposed that can be networked together using a particular type of network using a standard communication protocol layer, e.g., IEEE 1394 communication standard. The IEEE 1394 standard is an international standard for implementing an inexpensive high-speed serial bus architecture which supports both asynchronous and isochronous format data transfers. The IEEE 1394 standard provides a high-speed serial bus for interconnecting digital devices thereby providing universal input/output connection. The IEEE 1394 standard defines a digital interface for applications thereby eliminating the need for an application to covert digital data to an analog form before it is transmitted across the bus. Correspondingly, a receiving application will receive digital data from the bus, not analog data and will therefore not be required to convert analog data to digital form. The IEEE 1394 is ideal for consumer electronics communication in part because devices can be added to or removed from the serial bus while the bus is active. If a device is so added or removed, the bus automatically reconfigures itself for transmitting data between the then existing devices. Each device on the bus is a xe2x80x9cnodexe2x80x9d that has its own address on the bus and contains its own address space. Each node connected to the bus is capable of communicating with any other node also on the bus.
The provision of the IEEE 1394 serial communication bus for networking consumer electronic devices has introduced a powerful new platform on which device functionality and inter-operability can be built. For instance, in such a system, complex operations involving media transfers, media recordings and media presentation can be performed that involve two or more devices acting in concert. However, interaction between these devices can be complex, error prone and laborious if it is required that each device be directly touched in order to properly configure the desired operation. The problems associated with properly configuring the media operation can be exacerbated if one or more of the devices are remotely located and/or need constant attention and or adjustment. What is needed is an effective mechanism for interfacing with networked consumer electronic devices to facilitate media operations between two or more devices.
The physical separation between devices in a home AV network introduces a problem regarding control of a particular device. A user may be trying to remotely control a device that is physically separated from the user and thus is not able to be seen. For example, a VCR on the home AV network may be in one room, while the user is in another room containing the television. Therefore, the user is not sure whether he/she entered in the proper commands and that the device is correctly performing the desired function.
Another problem is introduced when there are multiple users simultaneously on the home AV network, each user working from a separate location. Thus, continuing the example above, more than one user may be attempting to control the VCR. Each user, not aware of the fact that there is another user also trying to use the VCR, will be entering commands that are perhaps conflicting and that are not being carried out properly. In addition, each user will not have information regarding the status of the VCR (e.g., whether the VCR is recording or playing), and thus will not be aware of whether they successfully entered the proper commands and whether the device is performing the desired function.
In addition, it is often desired to provide remote control access to the features performed by a consumer electronic device so that these devices can be accessed from a central location within a home or office. However, not all consumer electronic devices are able to provide a sophisticated display system for remote interaction, e.g., some devices offer only a small liquid crystal display (LCD) or a small collection of light emitting diodes (LEDs) as display devices. What is needed is a mechanism for interfacing remotely with devices that provides a sophisticated level of user interaction for devices of differing local display capability. Also, in many consumer electronic systems, each device can have its own remote control unit. In systems having even a modest amount of devices, the user discovers that three or more different and bulky remote control devices are required to perform media operations. What is needed is a mechanism for interfacing with electronic devices that reduces the problems of having different remote control units for different devices. What is needed further is a mechanism for interfacing with electronic devices that is flexible and can adapt to new devices and device types within the consumer electronics market.
Accordingly, the present invention provides an effective mechanism for interfacing with networked consumer electronic devices to facilitate media operations between two or more devices. The present invention also provides a mechanism for interfacing remotely with devices and that provides a sophisticated level of user interaction for many devices that themselves may have limited display capability. The present invention also provides a mechanism for interfacing with electronic devices that operates using a remote controller and a central display for instructing different electronic devices and for receiving status information regarding the different electronic devices. The present invention provides an interface mechanism for interfacing with electronic devices that is additionally flexible and can adapt to new devices and device types within the consumer electronics market. The present invention provides the above advantageous features within one embodiment that is compliant with the Home Audio/Visual Interoperability (HAVi) architecture. These and other advantages of the present invention not specifically mentioned above will become clear within discussions of the present invention presented herein.
A method and system are described herein for providing a user interface for a networked electronic device using a data driven interface (DDI) including a data structure of information maintained between a DDI target device and a DDI controller. The present invention utilizes the DDI to allow any compliant device (e.g., a DDI target device) to have the physical appearance of its controls and displays (e.g., elements) represented by a data structure used by another device (e.g., a DDI controller) and allows the DDI controller to thereby trigger actions on the DDI target device as if a user had directly physically manipulated the controls on the DDI target device. The present invention operates within a network of consumer electronic devices, e.g., television (TV), set-top-box, video cassette recorder (VCR), compact disk (CD) device, personal computer system (PC), etc., that are coupled together using a standard communication protocol layer, e.g., the IEEE 1394 serial communication standard. A DDI controller (e.g., a TV or set-top-box or computer system or other intelligent electronic device) monitors the network to discover the units coupled thereto and is programmed to locate DDIs that are defined for a DDI target device. The DDI target device may be remotely located. In one embodiment, the network is compliant with the HAVi architecture.
In general, the DDI provides the DDI controller with information for rendering a depiction of the controls and displays of the DDI target device and this depiction can be interacted with by a user for remotely triggering actions by the DDI target device. The DDI is defined such that command interpretations and image alterations are controlled by the DDI target device thereby reducing the responsibility of the DDI controller in managing the user interface with generic user events. This approach allows increased flexibility to expand the user interface and adjust to future control types and display information protocols.
The DDI uses graphical user interface (GUI) xe2x80x9celementsxe2x80x9d stored in a data structure to define the physical controls of the DDI target device. The GUI elements include standard types of controls and displays (e.g., push buttons, toggles, sliders, dials, LCD screens, alphanumeric inputs, etc.) that are commonly found on consumer electronic devices. The control types have well defined behaviors (e.g., buttons are pushed, dials are moved, values are input, etc.). In addition, the DDI defines a set of xe2x80x9cuser eventsxe2x80x9d which can be applied to any of these controls by the user. The user events are defined to be sufficiently generic so that they apply to most types of controls. The purpose of these generic user events is to encapsulate the typical user manipulated action for the GUI elements and to let the DDI target device decide the manner in which to interpret the user actions on its own thereby relieving the DDI controller of these responsibilities. Related elements of a DDI target device can be organized together into logical groups which the DDI allows to be displayed or processed together in special ways. The DDI controller is responsible for the exact visual appearance of the GUI elements and their exact layout on its display.
The internal state of the DDI target is maintained by the DDI target which keeps the DDI controller informed of its status. The DDI target forwards status update messages (upon changes of state) to the DDI controller to update its graphical display for the user. A status notification commands also allows the DDI controller to poll the DDI target for changes in state.
A particular embodiment of the present invention operates within the HAVi architecture. Using the DDI mechanism, a HAVi software element can provide a user with the ability to control another software element. Within this interaction, the first software element is termed the DDI controller and the second software element the DDI target. The DDI controller uses a description of the User Interface (UI) to be presented to the user and is obtained from the DDI target. The means by which this control is accomplished is described herein.
In this particular embodiment, the DDI controller and its DDI target are both HAVi components each executing on an IAV or FAV device. The components may be on the same or different devices, implemented using native code or Java bytecode; in all cases, though, they interact by sending HAVi messages to each other. The DDI controller communicates with the user by using the input and output devices of (typically) the device upon which the controller is executing. This input/output communication can be done in a controller-implementation-dependent manner. The DDI target can be a DCM that controls its device in a implementation-dependent manner. The HAVi DDI protocol described herein is structured so that a DDI controller may be written in a generic manner. That is, a DDI controller does not have to be implemented with knowledge of a particular DDI target in mind; all target-dependencies are represented in the DDI data provided by the target to the controller.